Often when interpreting a set of sampled data points the goal is to find the precise location of the peak value of the underlying process with accuracy greater than that supported by the spacing of the sample points. One-dimensional examples of this include interpolation of measurements as in radar and sonar location algorithms. Two-dimensional applications include location of a target in a scene image by image correlation, estimation of an optical wave front in an adaptive optics (AO) sensor, determining the precise location of a star in an optical star tracker, and determining the precise registration offset between two images. FIG. 1 shows an example result of a single digital correlation (e.g., it describes the estimated displacement of a vibrated test pattern). The precise registration or offset between the underlying image objects is estimated by determining the location of the center of the correlation peak.
For many processes involving the overlay of multiple layers of image data, the first step is often to register the images to one another so that they share an x-y addressing space. Image registration is often accomplished at many small, evenly spaced regions, or templates. The result of this process forms an optical flow matrix, which maps the estimated translation for each template. This is a map of the translations needed to warp one image into the other (assuming that the registration estimates at each point are good ones). The optical flow methodology, e.g., utilizing a map of translations to warp one image into another, is particularly useful when the difference between the images is more complex than a simple translation (e.g., global estimate). Examples of where a global estimate alone is inadequate and an optical flow method is useful include estimating an affine transformation, estimating range information through parallax, and motion detection and estimation. In general, precise sub-pixel estimation of optical flow is more challenging than estimation of global offset because the regions across which the registration measurements are made are much smaller.
Two methods commonly used to estimate the inter-pixel location of the peak value on a curve or surface are centroiding and curve fitting. In centroiding, a weighted average of the sample values near the peak location is used to find the center-of-mass of the peak. In curve fitting, the data points near the peak are approximated by a mathematical curve or surface with a known center point. Hyperbolas, parabolas, and Gaussian peaks are common forms for this fitting, and a least squares estimation algorithm is commonly used to find the best match between the curve or surface parameters and the measured data.
These methods are subject to inaccuracies for a number of reasons. Chief among these are: the data is noisy; a limited number of points are used for the estimate; and the real-world signal is not a perfect match to the mathematical surface used for fitting. These inaccuracies give rise to estimation biases and offsets as described in Grycewicz, T. J., Florio, C. J., Franz, G. A., and Robinson, R. E., “Estimation bias from using nonlinear Fourier plane correlators for sub-pixel image shift measurement and implications for the binary joint transform correlator,” Proc. SPIE 66950J (2007) and the references cited therein, which are incorporated herein by reference in their respective entireties.
When correlation-based processing is used to estimate the sub-pixel registration distance between two images, the result will include systematic sub-pixel biases. When the template is small (a few hundred pixels or fewer) a strong bias will be seen towards the trial location (toward an assessment that the template is perfectly aligned with the image, irrespective of actual alignment). For larger templates, the strongest bias is frequently toward the nearest perfect integer match to the actual sub-pixel displacement. These bias errors are unavoidable consequences of attempting to interpolate values on a correlation surface generated from pixilated images. The spatial quantization of the input image to a correlator and the pixellation in the correlation plane both place limits on the accuracy of interpolating the peak's location in the correlation plane. Moreover, the width and shape of the correlation peak can be a limiting factor to location interpolation. If the peak is either too large or too small for the area over which the peak center is being estimated, the result will show a systematic bias toward the center of the pixel. Although bias terms can be minimized or reduced by choosing an interpolation routine matched to the sharpness of the correlation peak, because correlation peak sharpness is determined by image content and can vary widely for typical images, such an interpolation process is difficult to implement and computationally expensive.
Image registration is the first step in many image processing processes such as change detection, motion estimation, jitter compensation, image mosaic stitching, image overlay processing, three-dimensional extraction from a series of two-dimensional images, pattern recognition, machine vision, and target detection/identification. Image registration techniques which can reliably register two images to the nearest pixel are well developed, but most techniques break down when sub-pixel accuracy is required. For correlation-based methods, the error usually manifests as an image dependent systematic bias of the registration measurement towards the nearest whole-pixel shift. However, for many tasks, such as analysis of polarimetric images and change detection, sub-pixel registration errors result in objectionable artifacts in the results.
It would be useful to be able to improve the accuracy of methods and systems for estimating peak location on a sampled surface. In image correlation applications, it would be useful to be able to improve correlation peak location accuracy. In registration applications, it would be useful to be able to decrease sub-pixel registration estimate errors caused by biases or artifacts from an interpolation process.